An Exploration Of The Effects Of Enhanced Compiler Error Messages For Computer Programming Novices

Computer programming is an essential skill that all computing students must master and is increasingly important in many diverse disciplines. It is also difficult to learn. One of the many challenges novice programmers face from the start are notoriously cryptic compiler error messages. These report details on errors made by students and are essential as the primary source of information used to rectify those errors. However these difficult to understand messages are often a barrier to progress and a source of discouragement. A high number of student errors, and in particular a high frequency of repeated errors – when a student makes the same error consecutively – have been shown to be indicators of students who are struggling with learning to program. This instrumental case study research investigates the student experience with, and the effects of, software that has been specifically written to help students overcome their challenges with compiler error messages. This software provides help by enhancing error messages, presenting them in a straightforward, informative manner. Two cohorts of first year computing students at an Irish higher education institution participated over two academic years; a control group in 2014-15 that did not experience enhanced error messages, and an intervention group in 2013-14 that did. This thesis lays out a comprehensive view of the student experience starting with a quantitative analysis of the student errors themselves. It then views the students as groups, revealing interesting differences in error profiles. Following this, some individual student profiles and behaviours are investigated. Finally, the student experience is discovered through their own words and opinions by means of a survey that incorporated closed and open-ended questions. In addition to reductions in errors overall, errors per student, and the key metric of repeated error frequency, the intervention group is shown to behave more cohesively with fewer indications of struggling students. A positive learning experience using the software is reported by the students and the lecturer. These results are of interest to educators who have witnessed students struggle with learning to program, and who are looking to help remove the barrier presented by compiler error messages. This work is important for two reasons. First, the effects of error message enhancement have been debated in the literature – this work provides evidence that there can be positive effects. Second, these results should be generalisable at least in part, to other languages, students and institutions

Half Empty, Half Full - An Examination of Subtractive Versus Additive Assessment

In this paper, we present a case study comparing subtractive marking schemes with the more common technique of additive marking. Although often accepted for use in oral language and multiple choice exams, subtractive marking has yet to see serious consideration as an accepted technique for assessing other more common paper exams, particularly Computer Science exams with subjective questions. This paper is presented in a number of sections. After a brief overview of previous investigations in the area, we outline an experiment conducted with real examination papers taken by students at Griffith College Dublin. In this experiment, we mark the same set of papers using both additive and subtractive schemes. We then summarise the differences between the two techniques and identify some of the challenges, advantages and disadvantages of subtractive approaches and also the motivations behind them. We also examine how different types of exam questions affect the difference between additive and subtractive marking and make the argument in favour of subtractive marking as a useful QA technique. Finally, we present the results of a student survey regarding their opinions on subtractive marking in order to gain an insight as to how students feel about the concept and what types of students feel strongly for or against it

Half Empty, Half Full - An Examination of Subtractive Versus Additive Assessment

In this paper, we present a case study comparing subtractive marking schemes with the more common technique of additive marking. Although often accepted for use in oral language and multiple choice exams, subtractive marking has yet to see serious consideration as an accepted technique for assessing other more common paper exams, particularly Computer Science exams with subjective questions. This paper is presented in a number of sections. After a brief overview of previous investigations in the area, we outline an experiment conducted with real examination papers taken by students at Griffith College Dublin. In this experiment, we mark the same set of papers using both additive and subtractive schemes. We then summarise the differences between the two techniques and identify some of the challenges, advantages and disadvantages of subtractive approaches and also the motivations behind them. We also examine how different types of exam questions affect the difference between additive and subtractive marking and make the argument in favour of subtractive marking as a useful QA technique. Finally, we present the results of a student survey regarding their opinions on subtractive marking in order to gain an insight as to how students feel about the concept and what types of students feel strongly for or against it

CSLINC: A Nationwide CS MOOC for Second-level Students

This poster introduces CSLINC, a free scaffolded MOOC framework tailored to second-level students in Ireland that consists of: an online platform built for accessibility; a suite of modules developed upon international best practices with varying co-creators; and automated assessment and certificates of completion. Its aim is to provide content to promote national CS curricula to all second-level students in Ireland. In September 2021, CSLINC launched to 10,000 students across 100 schools. Future work will include collecting and collating research to validate CSLINC’s goals, scaffolding that will build foundations for national curriculum learning outcomes, and measure its impact on students, their perceptions and follow on CS uptake at second-level in Ireland.https://arrow.tudublin.ie/cddpos/1012/thumbnail.jp

The European Commission and AI: Guidelines, Acts and Plans Impacting the Teaching Of AI and Teaching With AI

Recent developments, guidelines, and acts by the European Commission have started to frame policy for AI and related areas such as ML and data, not only for the broader community, but in the context of education specifically. This poster presents a succinct overview of these developments. Specifically, we look to bring together all publications that might impact the teaching of AI (for example, teacher expectations in the coming years around AI competencies) and publications that affect the use of AI in the classroom. We mean using tools and systems that incorporate both ‘Good Old Fashioned’ AI and those that can directly impact students. This poster is of value to both the European and the wider CER communities and practitioners, as it brings together several guidelines, acts, and plans that are not easily searchable or linked. The publications presented in this poster will impact the teaching of AI and teaching with AI in Europe, and insights can be drawn and compared for other jurisdictions as the educational world adapts to and with AI.https://arrow.tudublin.ie/cddpos/1025/thumbnail.jp

Adaptive Basketball Shooter - Final Project Report

The Friday Club is a joint venture between the Cal Poly Kinesiology Department and the San Luis Obispo Special Olympics that offers people with varying degrees of disability the opportunity to meet weekly and learn various sports and games. At Friday Club, athletes in wheelchairs with limited arm strength use devices built by Cal Poly mechanical engineering students in order to participate in various sports. Many devices are designed to attach to the Universal Play Frame (UPF), a wheel-chair attachment. The purpose of this project was to design and build a UPF device that will launch a basketball, so that an athlete can participate in a game of “Horse.” The project was worked on over the course of the 2013-2014 school year. To begin, the team developed a list of objectives for the device to meet and researched existing solutions for various facets of our design. The next step was to generate concepts of our device, and using Pugh matrices, proof of concept testing, and debate to narrow down to a single concept. Next, this concept was transformed into a fully-fledged design backed by engineering analysis. After design approval, all necessary parts and materials were ordered and a prototype was built over a 10 week period. The final prototype was tested with the Friday Club and displayed at the Senior Project Expo on May 31, 2014. The final device that our team designed is a slingshot that launches the ball by releasing stretched elastic bands. Our design attaches to the UPF at two points, and can be aimed to shoot a basket from anywhere between 5 and 15 feet away. The athlete has the ability to control the direction, power, and release of each shot. The device can be set up in under five minutes by a single person and takes only 30 seconds to reset between shots. Of all customer requirements that the device was to meet, it only failed to meet one of them. The first was that it should be able to shoot a three-pointer. Unfortunately, our device either did not have strong enough elastic, or did not have enough space to pull back the ball and carriage sufficiently. Therefore, our device can shoot a basket from a maximum of only 15 feet away, or a free throw. Other future recommendations are to strengthen some parts that take high impacts and to reduce the weight of some unnecessarily bulky parts. Our budget for this project was $1,500 but over the course of this project our team spent a little over$1,750. After analyzing our spending, we found that over $300 was spent because of manufacturing mistakes and mid-construction design changes. If we were to build another device with the exact same design as our final prototype, and with no mistakes, the device would cost about$1,400. In this report, our team’s entire design process is cataloged in detail. Also enclosed are detailed part drawings for each manufactured part of our final device, as well as a safety and operation manual

Assessing the Influencing Factors on the Accuracy of Underage Facial Age Estimation

Swift response to the detection of endangered minors is an ongoing concern for law enforcement. Many child-focused investigations hinge on digital evidence discovery and analysis. Automated age estimation techniques are needed to aid in these investigations to expedite this evidence discovery process, and decrease investigator exposure to traumatic material. Automated techniques also show promise in decreasing the overflowing backlog of evidence obtained from increasing numbers of devices and online services. A lack of sufficient training data combined with natural human variance has been long hindering accurate automated age estimation -- especially for underage subjects. This paper presented a comprehensive evaluation of the performance of two cloud age estimation services (Amazon Web Service's Rekognition service and Microsoft Azure's Face API) against a dataset of over 21,800 underage subjects. The objective of this work is to evaluate the influence that certain human biometric factors, facial expressions, and image quality (i.e. blur, noise, exposure and resolution) have on the outcome of automated age estimation services. A thorough evaluation allows us to identify the most influential factors to be overcome in future age estimation systems

Computer Science Outreach to Inform Secondary School Students’ Perceptions of Computer Science: Preliminary Findings

This poster describes a longitudinal K-12 outreach programme to promote Computer Science in Ireland, which ran over a three-year period from 2017- 2020. A pilot phase was conducted in the first year from 2017-2018 with 2900 students participating. The implementation phase began in 2018, when 7320 students participated across the 2018-2019 and 2019-2020 academic years. The programme consisted of a free onsite school delivery of a two-hour camp that introduced students to a range of Computing topics: addressing computing perceptions, introduction to coding, and exploration of computational thinking. Schools self-selected, and the programme reached a large number of schools with varied socio-economic and gender diversity, along with schools across every county in Ireland. The student ages ranged from third class (7 - 8 years old) in primary school to sixth year in second level (17 - 18 years old). This poster focuses specifically on the research data collected during the implementation phase (N=1211) from secondary school students (12 - 18 years old) not enrolled in the formal Leaving Certificate Computer Science subject. Looking at student perceptions of Computer Science and if the outreach positively impacted those perceptions and built student interest in pursuing further study in Computer Science.https://arrow.tudublin.ie/cddpos/1011/thumbnail.jp

On Designing Programming Error Messages for Novices: Readability and its Constituent Factors

The 2021 ACM CHI Virtual Conference on Human Factors in Computing Systems (CHI'21), Virtual Conference, 8-13 May 2021Programming error messages play an important role in learning to program. The cycle of program input and error message response completes a loop between the programmer and the compiler/interpreter and is a fundamental interaction between human and computer. However, error messages are notoriously problematic, especially for novices. Despite numerous guidelines citing the importance of message readability, there is little empirical research dedicated to understanding and assessing it. We report three related experiments investigating factors that influence programming error message readability. In the first two experiments we identify possible factors, and in the third we ask novice programmers to rate messages using scales derived from these factors. We find evidence that several key factors significantly affect message readability: message length, jargon use, sentence structure, and vocabulary. This provides novel empirical support for previously untested long-standing guidelines on message design, and informs future efforts to create readability metrics for programming error messages

Prompt Problems: A New Programming Exercise for the Generative AI Era

Large Language Models (LLMs) are revolutionizing the field of computing education with their powerful code-generating capabilities. Traditional pedagogical practices have focused on code writing tasks, but there is now a shift in importance towards code reading, comprehension and evaluation of LLM-generated code. Alongside this shift, an important new skill is emerging -- the ability to solve programming tasks by constructing good prompts for code-generating models. In this work we introduce a new type of programming exercise to hone this nascent skill: 'Prompt Problems'. Prompt Problems are designed to help students learn how to write effective prompts for AI code generators. A student solves a Prompt Problem by crafting a natural language prompt which, when provided as input to an LLM, outputs code that successfully solves a specified programming task. We also present a new web-based tool called Promptly which hosts a repository of Prompt Problems and supports the automated evaluation of prompt-generated code. We deploy Promptly for the first time in one CS1 and one CS2 course and describe our experiences, which include student perceptions of this new type of activity and their interactions with the tool. We find that students are enthusiastic about Prompt Problems, and appreciate how the problems engage their computational thinking skills and expose them to new programming constructs. We discuss ideas for the future development of new variations of Prompt Problems, and the need to carefully study their integration into classroom practice.Comment: Accepted to SIGCSE'24. arXiv admin note: substantial text overlap with arXiv:2307.1636